Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection

publishedVersio

Deep Learning-Based Framework for <i>In Vivo</i> Identification of Glioblastoma Tumor using Hyperspectral Images of Human Brain

The main goal of brain cancer surgery is to perform an accurate resection of the tumor, preserving as much normal brain tissue as possible for the patient. The development of a non-contact and label-free method to provide reliable support for tumor resection in real-time during neurosurgical procedures is a current clinical need. Hyperspectral imaging is a non-contact, non-ionizing, and label-free imaging modality that can assist surgeons during this challenging task without using any contrast agent. In this work, we present a deep learning-based framework for processing hyperspectral images of in vivo human brain tissue. The proposed framework was evaluated by our human image database, which includes 26 in vivo hyperspectral cubes from 16 different patients, among which 258,810 pixels were labeled. The proposed framework is able to generate a thematic map where the parenchymal area of the brain is delineated and the location of the tumor is identified, providing guidance to the operating surgeon for a successful and precise tumor resection. The deep learning pipeline achieves an overall accuracy of 80% for multiclass classification, improving the results obtained with traditional support vector machine (SVM)-based approaches. In addition, an aid visualization system is presented, where the final thematic map can be adjusted by the operating surgeon to find the optimal classification threshold for the current situation during the surgical procedure

Spatio-spectral classification of hyperspectral images for brain cancer detection during surgical operations - Fig 7

<p>Mean and variances of the pre-processed spectral signatures of the tumor, normal and blood vessel classes of the labeled pixels from patient 1 (A) and patient 2 (B), represented in red, black and blue color respectively.</p

Results of each step of the proposed cancer detection algorithm applied to the five different patients.

<p>(A), (B), (C), (D) and (E) Segmentation maps generated using the HKM algorithm. (F), (G), (H), (I) and (J) MV classification maps. (K), (L), (M), (N) and (O) OMD maps that take into account only the major probability per class obtained from the MV algorithm. (P), (Q), (R), (S) and (T) TMD maps that take into account the first three major probabilities per class obtained from the MV algorithm.</p

Processing time results comparison for each patient.

<p>Processing time results comparison for each patient.</p

Brain cancer detection and delimitation algorithm overview diagram.

<p>(A) HS cube of in-vivo brain surface. (B) Pre-processing stage of the algorithm. (C) Database of labeling samples generation. (D) SVM model training process employing the labeled samples dataset. (E), (F) and (G) Algorithms that conform the spatial-spectral supervised classification stage. (H) and (I) Algorithms that generate the unsupervised segmentation map and the final HELICoiD TMD map, respectively.</p

Hybrid classification example based on a majority voting technique.

<p>The unsupervised segmentation map and the supervised classification maps are merged using the majority voting method.</p

Quantitative results of the supervised classification performed with the SVM classifier applied to the labeled data of each patient.

<p>(A) Overall accuracy results of supervised classification per SVM kernel type and patient. (B) and (C) Specificity and sensitivity results obtained using the SVM classifier with linear kernel for each patient and class employing the One-vs-All evaluation method.</p

Results of each step of the optimized spatial-spectral supervised classification of the five different patients.

<p>(A), (B), (C), (D) and (E) Synthetic RGB images generated from the HS cubes. (F), (G), (H), (I) and (J) Golden standard maps used for the supervised classification training. (K), (L), (M), (N) and (O) Supervised classification maps generated using the SVM algorithm. (P), (Q), (R), (S) and (T) FR-t-SNE one band representation of the HS cubes. (U), (V), (X), (Y) and (Z) Spatially optimized classification maps obtained after the KNN filtering.</p